1Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University, Guilin 541004, China
2Guilin Center for Agricultural Science & Technology Research, Guilin 541006, China
3Agricultural Information Institutes of Science and Technology, Shanghai Academy of Agriculture Sciences, Shanghai 201403, China
4Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin, 541004, China
| Received 17 Nov 2022 |
Accepted 20 Aug 2023 |
Published 08 Sep 2023 |
Accurate detection and reconstruction of branches aid the accuracy of harvesting robots and extraction of plant phenotypic information. However, the complex orchard background and twisting growing branches of vine fruit trees make this challenging. To solve these problems, this study adopted a Mask Region-based convolutional neural network (Mask R-CNN) architecture incorporating deformable convolution to segment branches in complex backgrounds. Based on the growth posture, a branch reconstruction algorithm with bidirectional sector search was proposed to adaptively reconstruct the segmented branches obtained by an improved model. The average precision, average recall, and F1 scores of the improved Mask R-CNN model for passion fruit branch detection were found to be 64.30%, 76.51%, and 69.88%, respectively, and the average running time on the test dataset was 0.75 s per image, which is better than the compared model. We randomly selected 40 images from the test dataset to evaluate the branch reconstruction. The branch reconstruction accuracy, average error, average relative error of reconstructed diameter, and mean intersection-over-union (mIoU) were 88.83%, 1.98 px, 7.98, and 83.44%, respectively. The average reconstruction time for a single image was 0.38 s. This would promise the proposed method to detect and reconstruct plant branches under complex orchard backgrounds.
